Stochastic Complexity of Bayesian Networks

Bayesian networks arc now used in enormous fields, for example, system diagnosis, data mining, clustcrings etc. In spite of wide range of their applications, the statistical proper­ tics have not yet been clarified because the models are nonidentifiable and non-regular. In a Bayesian network, the set of parame­ ters for a smaller model is an analytic set with singularities in the parameter space of a large model. Because of these singulari­ ties, the Fisher information matrices are not positive definite. In other words, the mathe­ matical foundation for learning has not been constructed. In recent years, however, we have developed a method to analyze non­ regular models by using algebraic geometry. This method revealed the relation between model's singularities and its statistical prop­ erties. In this paper, applying this method to Bayesian networks with latent variables, we clarify the orders of the stochastic complexi­ ties. Our result shows that their upper bound is smaller than the dimension of the parame­ ter space. This means that the Bayesian gen­ crali2ation error is also far smaller than that of a regular model, and that Schwarz's model selection criterion BIC needs to be improved for Bayesian networks.